Adaptive two-layer ReLU neural network: I. Best least-squares approximation

In this paper, we introduce adaptive network enhancement (ANE) method for the best least-squares approximation using two-layer ReLU neural networks (NNs). For a given function f(x), ANE generates NN and numerical integration mesh such that accuracy is within prescribed tolerance. The provides natural process obtaining good initialization which crucial training nonlinear optimization problems. Numerical results functions of two variables exhibiting either intersecting interface singularities or sharp interior layers demonstrate efficiency method.